Balanced Perception and Action in the Tactical Language Training System

The goal of the Tactical Language Training (TLT) system is to rapidly teach people how to speak in a foreign language and behave in a foreign culture. TLT combines an intelligent tutoring system with a 3D game where learners get to practice their skills in simulated social situations. In the tutoring system, the learners interact with a tutoring agent that can listen to their pronunciation and give appropriate feedback, while in the game they have conversations with characters they meet using speech and gesture. The paper describes TLT as work in progress and discusses how balanced perception and action is addressed in both the tutoring agent and the interactive characters, focusing on a novel approach to speech perception and feedback generation that specifically targets learner speech.